Dimensionality reduction by local processing

In this paper we describe a novel approach towards dimensionality reduction of patterns to be classi ed. It consists of local processing of the patterns as an alternative to the well-known global principal component analysis (PCA) algorithm. We use a feed-forward neural network architecture with spatial or spatio-temporal receptive eld connections between the rst two layers that yields a transformed feature vector of signi cantly reduced dimension. We suggest two techniques to adapt the weights of the receptive elds: a local PCA algorithm and training by online gradient descent. Our dimensionality reduction algorithm requires computational costs that are several times smaller compared to the classical PCA approach without loosing performance in the subsequent classi cation process. We apply the algorithm to the problem of handwritten digit recognition as well as to the recognition of pedestrians in image sequences.